All of the material in this patent application is subject to copyright protection under the copyright laws of the United States and of other countries. As of the first effective filing date of the present application, this material is protected as unpublished material.
However, permission to copy this material is hereby granted to the extent that the copyright owner has no objection to the facsimile reproduction by anyone of the patent documentation or patent disclosure, as it appears in the United States Patent and Trademark Office patent file or records, but otherwise reserves all copyright rights whatsoever.
1. Field of the Invention
The invention disclosed broadly relates to the field of image processing and more specifically to image processing in a digital camera for taking panoramic pictures.
2. Description of the Related Art
Today, panoramic photography is accomplished in various ways. One is to use a still camera mounted on a tripod to take a succession of shots as the camera is pivoted around the tripod. In some cameras, a wider than usual strip of film is exposed with special movable optics.
In other cameras, conventional format film, such as 35 mm film, is masked during the exposure in the camera to provide a panoramic effect. The effect is panoramic but the whole exposure is limited by the field of view through the lens.
Other techniques for creating panoramic photography include to physically cut and paste together strips of exposed film by carefully aligning boundaries between edges of film.
The benefits of electronic photography have led to the growth of digital cameras, that, unlike their film-based counterparts, store images captured in memory into digital memory such as flash memory. To provide panoramic photography effects, these digital cameras can interface with personal computers for joining together two or more images into one image to provide a panoramic effect by joining edge boundaries of images. One such system is disclosed in U.S. Pat. No. 6,682,197, by named inventors Omid A. Moughadam, Stuart R. Ring, and John R. Squilla, entitled xe2x80x9cElectronic Panoramic Camera For Use With An External Processor.xe2x80x9d
Complicated panoramic digital cameras are available that rely on position sensors or satellite communications for determining position coordinates. These position coordinates are used to help combine the panoramic images. The process of combining scenes taken from different camera orientations is known as xe2x80x9cImage Stitching.xe2x80x9d One such system is disclosed in U.S. Pat. No. 5,262,867 by named inventor Kiyonobu Kojima entitled xe2x80x9cElectronic Camera and Device for Panoramic Imaging and Object Searchingxe2x80x9d issued on Nov. 16, 1993.
A panoramic camera with a memory device for storing data from a previously photographed portion of an object and a control device for enabling the display device to substantially display both the image to be photographed and the image already photographed and stored in the memory space is described in U.S. Pat. No. 5,138,460 by named inventors Egawa and Akira entitled xe2x80x9cApparatus for forming Composite Imagesxe2x80x9d issued on Aug. 11, 1992.
Although these techniques are useful, they are not without their shortcomings. One shortcoming is the expense and complication arising from the need for orientation and position sensors for determining picture orientation. Accordingly, a need exists for a device and method to create panoramic pictures without the need and expense of position sensors.
Another shortcoming of the current panoramic cameras is how difficult it is to properly overlap a region between two adjacent frames of a panoramic image. Too much overlap results in wasting memory in the digital camera with redundant information.
Another shortcoming with current panoramic cameras is their inability to guide the user where the correct edge overlap is required for creating a panoramic picture alignment from two or more images. Accordingly, a need exists to overcome this problem and to guide the user of a camera using a method and apparatus to overlap two or more images to provide a correct panoramic picture.
Another shortcoming of the current panoramic cameras is the requirement to overlap regions of two or more images. It is common with panoramic image generation to stitch together two or more images. Problems at the boundaries of two images include color distortion, perspective distortion, pixel adjacency and skipped pixels at the edges when joining two or more images. Accordingly, a need exists for a method and apparatus to overcome these shortcomings.
Another shortcoming of the current panoramic cameras is their inability to integrate these mechanisms with other products. The bulk, expense, and complication make these designs difficult to integrate. Accordingly, a need exists for a method and apparatus to allow easy integration of panoramic capabilities into other electronic devices.
Another shortcoming is the inability to easily capture two or more images to create a panoramic scene without the need for expensive computational resources in the camera and without the need for expensive position sensors. Accordingly, a need exists for a method and apparatus to provide a camera which can capture two or more images so as to create a panoramic image.
Another shortcoming is the requirement of having two distinct buffers to transform the captured image from one coordinate system to another coordinate system, such as rectilinear coordinates to cylindrical coordinates. The expense and space required for two buffers for coordinate transformations is undesirable. Accordingly, a need exists to provide a method and apparatus that can transform images from one coordinate system to another coordinate system, without the use of two buffers.
Another shortcoming with current panoramic cameras is the perspective of the series of images relative to each other is lost. One known procedure uses sub-sampling. The picture is scaled down so that the horizontal dimension will match the horizontal extent of the view-port. However, the picture must become a narrow stripe in order to maintain the correct aspect ratio. This will produce a great loss of vertical resolution. Another solution is to scroll the panoramic image horizontally by a fixed amount. In this case no sub-sampling is used, provided that the vertical dimension of the picture will fit the vertical dimension of the view-port. Accordingly, a need exists to provide a method and apparatus to create a moving display of still pictures with perspective without the need to sub-sample or to horizontal scroll.
Still another shortcoming with current panoramic cameras is the requirement that all the image processing electronic circuitry for the final stitching together of one or more images in a series into a single panoramic image is integrated into the camera. Many times, the added expense, the additional weight and the additional size of the electronic circuitry make the camera unwieldy to carry and operate. Moreover, the additional electronic circuitry makes the camera more expensive to manufacture. Accordingly, a need exists for a method and apparatus to enable the preview of a series of adjacent images to form a panoramic, but to enable the final image stitching processing on a remote device such as a personal computer.
A method for converting a rectilinear image and a focal length into a cylindrical image parameterized by a height of the cylinder and an angular distance in a single buffer on a remote processing device. The method, on the remote processing device, comprising joining two or more images together to form a panoramic image with corrected perspective. The rectilinear transformation is xe2x80x9cin-placexe2x80x9d and requires only one buffer. Color correction and motion estimation is also carried out on the remote device.
In an alternate embodiment, a computer readable medium corresponding to the above method is described.